Majorization for CRFs and Latent Likelihoods

The partition function plays a key role in probabilistic modeling including condi-tional random fields, graphical models, and maximum likelihood estimation. Tooptimize partition functions, this article introduces a quadratic variational upperbound. This inequality facilitates majorization methods: optimization of com-plicated functions through the iterative solution of simpler sub-problems. Suchbounds remain efficient to compute even when the partition function involvesa graphical model (with small tree-width) or in latent likelihood settings. Forlarge-scale problems, low-rank versions of the bound are provided and outper-form LBFGS as well as first-order methods. Several learning applications areshown and reduce to fast and convergent update rules. Experimental results showadvantages over state-of-the-art optimization methods.